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Abstract 

In the case of ergodicity much of the structure of a one-dimensional time-discrete 
dynamical system is already determined by its ordinal structure. We generally dis- 
cuss this phenomenon by considering the distribution of ordinal patterns, which 
describe the up and down in the orbits of a Borel measurable map on a sub- 
set of the real numbers. In particular, we give a natural ordinal description of 
Kolmogorov-Sinai entropy of a large class of one-dimensional dynamical systems 
and relate Kolmogorov-Sinai entropy to the permutation entropy recently intro- 
duced by Bandt and Pompe. 

Keywords: time-discrete dynamical system, Kolmogorov-Sinai entropy, permutation en- 
tropy. 

1 Introduction 

The quantification of complexity is an important topic in the theory and application of 
dynamical systems. Various complexity measures have been introduced and studied, like 
the correlation dimension given by Grassberger and Procaccia [I] and the Kolmogorov- 
Sinai entropy (see [1Q]). Both are well- motivated quantities, but they are not easy to 
estimate from real data. Recently, Bandt and Pompe [3] have proposed to measure 
complexity of one-dimensional dynamical systems on the base of the distribution of ordinal 
patterns in the system. 

Given a one-dimensional time-discrete dynamical system, defined by a map on a subset of 
the real line, ordinal patterns of order d classify, roughly speaking, points according to the 
order type of the vectors consisting of the points and of their first d iterates. The higher 
d is, the more information is contained in the patterns. When the dynamical system is 
equipped with a probability measure which is invariant with respect to the given map, 
this measure yields a distribution on the finite set of ordinal patterns of some order. The 
permutation entropy introduced by Bandt and Pompe |3j is the Shannon entropy of the 
ordinal pattern distribution of order d relative to d for d — > oo and provides a relatively 
simple concept for measuring complexity in a 'standardized' way. It is remarkable that 
in the case of a piecewise monotone interval map the permutation entropy coincides with 
the Kolmogorov-Sinai entropy (see Bandt et al. [5]). Also note that a similar result has 
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been given for the topological entropy (see Bandt et al. [3] and, for some further results, 
Misiurewicz [7]). 

From the viewpoint of estimating complexity, ergodic dynamical systems are most impor- 
tant because according to the Birkhoff Ergodic Theorem many properties of such systems 
can be reconstructed from a single orbit with probability one. For interval maps and 
their multidimensional generalizations, Amigo et al. pQ show equality of a modification of 
the permutation entropy to the Kolmogorov-Sinai entropy. Their concept of permutation 
entropy uses a non-standardized sequence of finer and finer partitions of the considered 
interval as the Kolmogorov-Sinai entropy itself. 

In the following we discuss consequences of invariancy and ergodicity in a one-dimensional 
dynamical system for the ordinal structure of the system. On this base, we give an ordinal 
description of the Kolmogorov-Sinai entropy for ergodic systems and continuous systems 
on compact sets and investigate the relationship of permutation entropy and Kolmogo- 
rov-Sinai entropy. 

Ordinal patterns. The central concept of this paper is the concept of an ordinal pat- 
tern. We give it here in a slightly altered form which is more convenient for the following 
considerations. 

Definition 1. For d G N = {1, 2, 3, . . .}, we call X d : = Xf =1 {0, 1, . . . , 1} the set of ordinal 
patterns of order d. Further, for X C K. and a map / : X <— 3 the sequence i^ = i d {x) = 
(i d ,i(x),i dj2 (x), . . .,i d ,d(x)) G I d defined by 

(1) i d>l (x) = #{r G {0, 1, ...,/- 1} | f«*-*\x) < f°^ l \x)} 

for I = 1, 2, . . . , d is called the ordinal pattern of order d G N realized by x G X. Here 
f on (x) for n G N denotes the n-th iterate of a point x G X with respect to / where 
f o0 (x) = x. 

The vector i d (x) of inversion numbers just describes the order relations between the com- 
ponents of the vector (x, f(x), f o2 (x), . . . , f od (x)). Note that originally ordinal patterns 
were given in terms of permutations which express the order relations more directly than 
the inversion numbers (see |3J). However, the latter representation is better from the 
computational viewpoint (see [6]), and the main point is that both descriptions of ordinal 
patterns provide exactly the same partitions of the interested system. In particular, both 
representations contain the information on whether f ok (x) < f ol (x) for k, I G {0, 1, . . . , d} 
with I < k. 

Partitions and entropies. Let X C K, let / : X be a B(X)-B(X)-measurable map, 
where M(X) denotes the Borel a-algebra on X, and let \x be an f -invariant probability 
measure on B(X), that is 

for each A G B(Jf). For the following we refer to the simple fact that finite a-algebras 
on X are in one-to-one relation to finite partitions of X, since there is exactly one finite 
partition generating a finite a-algebra. 
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Recall the definition of the (Shannon) entropy of a finite partition A = {Ai,A 2 , 
. . . , Ak\ C M(X) of X and of the from A generated cr-algebra £>, respectively, given 
by 

k 

H(B) = H(A) = - y £»( A i)^»(A j ). 

i=i 

For a finite partition A C M(X) of X, let h^f^A) := lim,^ ±H(Y££ f~ oj {A)), 
where \/™Zo f ~ oj (A) denotes the cr-algebra generated by the set of all f~ i(A) with 
j G {0, 1, . . . , n — 1} and A G A. It is well known that this limit exists (see, e.g., [TO] . 
Cor. 4.9.1). The Kolmogorov-Sinai entropy of / is defined by 

h^f) := swp{h^(f,A) | A C B(X) finite partition of X}. 

The Kolmogorov-Sinai entropy is an important theoretical concept in ergodic theory, how- 
ever, its estimation from an orbit of a dynamical system, is complicated since principally 
arbitrarily fine partitions have to be considered. The permutation entropy introduced by 
Bandt and Pompe [3] seems to be an interesting alternative. 

Definition 2. Let X C R, let / : X be a B(X)-B(X)-measurable map, and let fi be 
an /-invariant probability measure on B(Jf). For d G N, let 

V d = {Pi | i G J4 with Pi = {s G X | = i}. 

The permutation entropy of / is defined by 

h;(f)=hmsup±H(V d ). 

d— >oo A 

Note that for each d G N and i G Z^, the set Pi indeed belongs to M(X). This immediately 
follows from the facts that {x G X | /°( d ~ r )(x) < /^^^(x)} = - f°( d - r )y\X n 

[0, oo[), the mapping - /°( d " r ) is B(X)-B(X)-measurable and X n [0, oo[ G B(X) 

(compare formula ([T])). 

2 Generating properties 

The key for proving the main results of the paper is that in the ergodic case ordinal 
patterns separate points, in other words, the partitions defined by ordinal pattern are 
'generating'. Note that finite generating partitions play an important role in ergodic 
theory (see [10J, §4.6). By considering ordinal patterns of increasing order we obtain 
a sequence of finite partitions Pi, 7-2,^3; • • • generating increasing cr-algebras, that is 
a{Vi) C a(P 2 ) C a(P 3 ) C . . .. 

The following lemma shows how ordinal information of a dynamical system with invariant 
measure can be extracted from its orbits. Recall that if X C R and / : X *-^> is M(X)- 
B(X)-measurable, then an /-invariant probability measure /i on M(X) is called ergodic 
if 

H(A) G {0, 1} for each A G M(X) with / _1 (A) = A. 
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It is well-known that this is equivalent to the statement that fj,(f 1 (B)AB) = for 
B G B(X) implies fx(B) G {0, 1}, where A denotes the symmetric difference of sets (see 
PD], Th. 1.5.). 

Lemma 3. Let X C R ; let f : X ^ be a M(X)-M(X) -measurable map, and let [i be an 
f -invariant probability measure on B(X). 

Then there exists a set X G B(X) with n(X) = 1 and f{X) C X, suc/i £/ia£ 

a(x) := lim d,d ^ - 

d^oo d 

is well-defined for all x G X . If fi is ergodic, X can be chosen such that moreover 
ol(x) = /J>({y G X | y < x}) and ct is injective on X . 

Proof. Let zGl, and for x G X let 

#{zG{l,2,...,4|r(x)<z} 
a(z, xj := lim — - 

if the limit exists. Fix a set X z G B(X) with fi(X z ) = 1 consisting of points x G X for 
which a;) exists. Moreover, in case of ergodicity of / require that fi({y G X\y < 
z}) = ct(z,x) for all x G X z . Both is possible by the Birkhoff Ergodic Theorem (see, 
e.g., [10], Th. 1.14. and text below). For X z : = f|„ gN f~° n {X z ) it holds f(X z ) C X z 
and, since \x is /-invariant, [i{X z ) = 1. Moreover, since ck(z, x) is defined if and only if 
ot(z, f(x)) is, cx(z, x) is defined for all x G X z . 

Let now Z be a countable dense subset of X containing all points of X isolated from the 
left or from the right, that is, every z£l with ]z — e, z[ H X = or ]z, z + e[ H X = 
for some e > 0. Let X = {~\ zeZ X z . Obviously, //(X) = 1, and /(X) C X. We show that 

ct(x, x) is defined for x G X. If x G X D Z, this is obvious. For the case where x G X \ Z, 
consider 

£ = { ] — oo, z] | z G Z} 
and let o~(£) denote the a-algebra on R generated by £. Obviously, 

:= lim #{ i£ {i.2,... d}ir W£B} 

d^oo d 

is well-defined for B & £ and can uniquely be extended to a probability measure on the 
measurable space (R, &(£)), which we still denote by P x . Now let (yk)kem be a mono- 
tonically increasing sequence in Z and (z^ken be a monotonically decreasing sequence 
in Z such that lim^oo yk = lim^oo Zk = x. (By definition of Z such sequences always 
exist.) By definition of X, the numbers a(yk,x),oc(zk,x) are defined for all k G N. Ob- 
viously, the sequences (cx(y k , x))k^n and (a^z*., x))k£N are monotonically nonincreasing 
and monotonically nondecreasing, respectively, therefore the limits lim^oo ct(yk, x) and 
linn^oo ct(zk, x) both exist. Further, P x ({x}) = lim^oo #{' g { 1 ' 2 '---.4 1 / W-^} - IS obviously 
well-defined. In particular, P x ({x}) = — if m is the least integer such that f om (x) = x 
and P x ({x}) = if no such integer exists. Hence 

a{y k ,x) + P x ({x}) < hmmf 

d^co a 

, y #{»£{l,2,...,4|f(x)<x} 

< hmsup < a{Zk,x) 

d— >oo U> 
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for every fceN. By the upper continuity of finite measures one obtains 

lim (ex(z k ,x) - (ex(y k ,x) + P x ({x}))) = lim P x Qy k , z k \) - P x {{x}) = 0. 

Consequently, lim^oo ^^ e ^ 1 ' 2 '---'^ I ( x ^ x } exists, and hence ex(x) = ex(x,x) is defined for 
each x & X and coincides with lim^oo /i({y G X \ y < z k }) = [i({y G X \ y < x}) for ji 
ergodic. 

In particular, ex is monotonically nondecreasing in the ergodic case. Therefore, if J = 
ex' 1 (a) consists of more than one point for some a G K, then J is the intersection of 
X with some interval, hence there are at most countable many of such a. In the case 
/i(J) > 0, necessarily, min J exists and ^(J) = /i({min J}). Otherwise, one would find 
some x,y G J with x < y such that /i( ] — oo, x] H J) < n( ] — oo, y] H J), leading to 

ct(x) — fj,() — oo, x) PI X) < — oo, y] H X) — a(y) 

in contradiction to ex being constant on J. Let 

U = [j{J\ J = ex-\a) for some a G M, #J > 1,/jl(J) = 0} 

U (J{J\ {min J} | J = ex' 1 (a) for some a G R, #J > l ;/ u(J) > 0} 

Clearly, /x(?7) = 0. By omitting all points with iterates in U the set X can be modified 
such that ex is injective. □ 

Remarks: 

1. The proof of the lemma is given under relatively general assumptions. If [i is ergodic, 
then ex is just the distribution function of /i, which can easily seen by the 'ergodic part' 
of the Birkhoff Ergodic Theorem (see, e.g., the remark below Th. 1.14. in [TO]) and the 
Clivenko-Cantelli argument (see, e.g., |9j, sec. II. 2.). It remains to show injectivity of ex 
on a set of full measure, which is the last part of our proof. The case that X is compact 
and / is continuous can reduced to the ergodic case argueing with ergodic decomposition 
(see e.g. Pj, §6.2.) 

2. Bandt and Shiha |2j have used the above statement for ergodic /i with continuous dis- 
tribution function, in order to show that all finite-dimensional distributions of a stationary 
stochastic process can be reconstructed from its ordinal structure and the one- dimensional 
distribution. 

As the above lemma shows, ergodic case ordinal patterns are 'generating' in the ergodic 
case: 

Corollary 4. Let IcR, let f : X be a M(X)-M(X) -measurable map, and let be an 
f -invariant ergodic probability measure on M(X). 

Then there exists a set X C X with n(X) = 1 and f(X) C X , such that different points 
of X have different ordinal patterns of some order d. 

We will need the supplement to Lemma [3] below, where \f deN Vd denotes the cr-algebra on 
X generated by \J deN V d - 



6 



Lemma 5. Let X be defined as in Lemma\^ ex be considered as a map on X, and V 
denote the restriction of \J d£N Vd to X. Then ex is V-M([0, 1}) -measurable. 

Proof. For a G [0, 1] and X a := {x G X \ ex(x) < a} it holds 

X a = {x G X | For all q G Q H ]x , oo[ there exists a d G N 

with - < q for all d > d } 
d 

= I f| U P) {x G X | z d , d (x) < dq} J n X. 

Since for each G N, q G Q , the set {x G X | id,d(x) < dq} belongs to the cr-algebra 
generated by V d , the set r\ 6 Qn]* ,°o[ Ud () eN fld>d { x e ^ I < d <?} is contained 

in \J d€N Vd- The system {]0, a],|a G [0,1]} generates B([0, 1]), and hence the result 
follows. □ 



3 Kolmogorov- Sinai entropy and order structure 

As well known, for a map / and an invariant measure fi the Kolmogorov-Sinai entropy can 
be obtained as lim^oo h^(f, A n ) if M(X) is generated by a sequence of finite partitions 
A n with corresponding increasing a-algebras (see [10], Th. 4.22). The following theorem 
says that under relative mild assumptions this sequence can be chosen in a standard way, 
namely by considering the partitions of X with respect to ordinal patterns of order d for 
increasing d. 

Theorem 6. Let X C M., let f : X be a M(X)-M(X) -measurable map, and let \x 
be an f -invariant probability measure on B(X). If fi is ergodic or X is compact and f 
continuous, then 

(2) K{f) = Jim h^f,V d ) = sup h^f,V d ). 

Proof. We start with the case that \i is ergodic. Clearly, \/ deN Vd is contained in M(X) 
(see end of Section [1]). Let X be chosen as for the ergodic case in Lemma [3] and let ex be 
considered as a map on X. Then ex is injective, monotone and, by LemmaO P-B([0, Im- 
measurable. Therefore, one easily sees that V coincides with the restriction of B(X) to 
X. Hence for each B G E(X) there exists an A G \/ den Vd with fj,(AAB) = 0, and §2§ 
follows from Theorem 4.22 in [ID] . 

In the case that /i is non-ergodic use ergodic decomposition (see, e.g., [ID], §6.2.) and the 
Monotone Convergence Theorem (see e.g. [ID], Theorem 0.8): fi can be written as /i = 
J E mdT(m), where E denotes the set of ergodic /-invariant Borel probability measures 
and r is a Borel measure on the /-invariant Borel probability measures on X (with respect 
to the weak* topology) with t(E) = 1. According to results shown by Jacobs (compare 
[ID] . Theorem 8.4), it holds 

(3) h^if, A)= I h m (f, A) dr(m) for each finite partition A C B(X) of X, 

Je 

(4) K{f)= I h m (f)dr(m). 

Je 



7 



By the already shown, the sequence (gdjdm of functions g d on the /-invariant Borel 
probability measures defined by gdijn) = h m (f,Vd) on E and vanishing outside of E 
is monotonically increasing and converges pointwisely to h m (f) on E. By the Monotone 
Convergence Theorem, either (h^f, V d ))deN is bounded and it holds (J2J), or (h^f, Vd))d<=N 
is unbounded and h m (f) is infinite on a set of positive measure with respect to r. In the 
latter case h^(f) = 00. □ 



4 Permutation entropy 

As a relatively simple consequence of the above theorem, we obtain that the Kolmogorov- 
Sinai entropy is not larger than the permutation entropy. This generalizes a result given 
by Amigo et al. [I] for ergodic maps / on intervals with finite h^(f). 

Corollary 7. Let X C R, let f : X be a B(X)-B(X) -measurable map, and let [x be 
an j ' -invariant probability measure on M(X). If [i is ergodic or X is compact and f is 
continuous, then h^{f) < hZ(f)- 

Proof. If h^f) = 0, we are done. Otherwise, consider some a > with h^f) > a. 
Given some c > 1 with > ca, by the above theorem there exists ado^N with the 

following property: For each d > do there exist some n{d) with ^i^(Vj=o /~°' 3 'C^ > d)) > ca 
for all n > n(d). 

Let d > do and n > max{n(d), -^f}- Note that the cr-algebra generated by "P^+n contains 
Vj=o f~°^(f > d)- This can be seen from the fact that for each i = (ii, i 2 , . . . , id) G Id and 
j G {0, 1, . . . , n — 1} one has 

d d 

r oj (Pi) = f]{xeX\ id.i (/ ■'(■'■)) = »,} = P|{x G X I i w (a;) = it}. 
1=1 1=1 

Therefore, one easily sees that / _OJ (Pi) is contained in the cr-algebra generated by sets 
{x G X I / ofc (x) < / oi (x)} with k, I G {0, 1, . . . , d + n} and I < k (compare to §B)). Since 
validity of f ok (x) < f (x) only depends on the ordinal pattern of x (see below definition 
of ordinal patterns), such sets are the union of sets in Vd+ n , hence the set f~°^(Pi) belongs 
to the ex-algebra generated by Vd+ n - 

Consequently, H(V d+Tl ) > i^(V™=o f~° J {Pd)), which implies 

1 1 n ~ x 

ITU £ IT-n^Vf-"™ 

> 7 — T^—myr^CPd)) 

(c — 1) n + n v 

v 1 ^ 3=0 



cn v 



3=0 
> a. 



From this one obtains h*(f) > a and, since a can be chosen arbitrarily near to h^f), it 
follows h*Jf) > h^f). □ 
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The central question under which assumptions Kolmogorov-Sinai entropy and permuta- 
tion entropy coincide remains open in the general case, whereas for piecewise monotone 
interval maps with invariant probability measure coincidence has been shown by Bandt 
et al. [5], as already mentioned in the Introduction. A map / from an interval [a, b] is said 
to be piecewise monotone if there are points Cq, c\, . . . , with a = c < C\ < . . . < Ck = b 
such that / is continuous and strictly monotone on [cj_i,Cj] for each j = 1,2,..., A;. 
(Continuity of the whole map is not required, in contrast to the often given definition of 
a piecewise monotone interval map.) Note that the proofs in [5] do not provide an idea 
for a generalization since they essentially base on piecewise monotonicity, in particular, 
on a result given by Misiurewicz and Szlenk [8] . 

As mentioned in the Introduction, Amigo et al. [1] have given a modified concept of permu- 
tation entropy of an interval map, for which they have shown equality to the Kolmogorov- 
Sinai entropy. Their idea is to consider permutation entropy for a stationary stochastic 
process with values in a finite set equipped with an arbitrary order. (The resulting en- 
tropy does not depend on the order.) Given a partition of the interval on which the map 
is defined into finitely many subintervals, one only regards in which of the subintervals a 
point lies. This yields the desired stochastic process, and the permutation entropy of the 
particular partition is naturally defined. The permutation entropy of the interval map 
is then defined as the limit of permutation entropies of partitions with interval lengths 
tending to 0. 

Just like the Kolmogorov-Sinai entropy, this modified concept of permutation entropy 
uses infinitely many partitions which in contrast to the original definition of permutation 
entropy are non-standardized. Note that the result of Amigo et al. p] is given not only 
for interval maps but in a multidimensional context. 
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1 Introduction 

The quantification of complexity is an important topic in the theory and application of 
dynamical systems. Various complexity measures have been introduced and studied, like 
the correlation dimension given by Grassberger and Procaccia [I] and the Kolmogorov- 
Sinai entropy (see [1Q]). Both are well- motivated quantities, but they are not easy to 
estimate from real data. Recently, Bandt and Pompe [3] have proposed to measure 
complexity of one-dimensional dynamical systems on the base of the distribution of ordinal 
patterns in the system. 

Given a one-dimensional time-discrete dynamical system, defined by a map on a subset of 
the real line, ordinal patterns of order d classify, roughly speaking, points according to the 
order type of the vectors consisting of the points and of their first d iterates. The higher 
d is, the more information is contained in the patterns. When the dynamical system is 
equipped with a probability measure which is invariant with respect to the given map, 
this measure yields a distribution on the finite set of ordinal patterns of some order. The 
permutation entropy introduced by Bandt and Pompe |3j is the Shannon entropy of the 
ordinal pattern distribution of order d relative to d for d — > oo and provides a relatively 
simple concept for measuring complexity in a 'standardized' way. It is remarkable that 
in the case of a piecewise monotone interval map the permutation entropy coincides with 
the Kolmogorov-Sinai entropy (see Bandt et al. [5]). Also note that a similar result has 
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been given for the topological entropy (see Bandt et al. [3] and, for some further results, 
Misiurewicz [7]). 

From the viewpoint of estimating complexity, ergodic dynamical systems are most impor- 
tant because according to the Birkhoff Ergodic Theorem many properties of such systems 
can be reconstructed from a single orbit with probability one. For interval maps and 
their multidimensional generalizations, Amigo et al. pQ show equality of a modification of 
the permutation entropy to the Kolmogorov-Sinai entropy. Their concept of permutation 
entropy uses a non-standardized sequence of finer and finer partitions of the considered 
interval as the Kolmogorov-Sinai entropy itself. 

In the following we discuss consequences of invariancy and ergodicity in a one-dimensional 
dynamical system for the ordinal structure of the system. On this base, we give an ordinal 
description of the Kolmogorov-Sinai entropy for ergodic systems and continuous systems 
on compact sets and investigate the relationship of permutation entropy and Kolmogo- 
rov-Sinai entropy. 

Ordinal patterns. The central concept of this paper is the concept of an ordinal pat- 
tern. We give it here in a slightly altered form which is more convenient for the following 
considerations. 

Definition 1. For d G N = {1, 2, 3, . . .}, we call X d : = Xf =1 {0, 1, . . . , 1} the set of ordinal 
patterns of order d. Further, for X C K. and a map / : X <— 3 the sequence i^ = i d {x) = 
(i d ,i(x),i dj2 (x), . . .,i d ,d(x)) G I d defined by 

(1) i d>l (x) = #{r G {0, 1, ...,/- 1} | f«*-*\x) < f°^ l \x)} 

for I = 1, 2, . . . , d is called the ordinal pattern of order d G N realized by x G X. Here 
f on (x) for n G N denotes the n-th iterate of a point x G X with respect to / where 
f o0 (x) = x. 

The vector i d (x) of inversion numbers just describes the order relations between the com- 
ponents of the vector (x, f(x), f o2 (x), . . . , f od (x)). Note that originally ordinal patterns 
were given in terms of permutations which express the order relations more directly than 
the inversion numbers (see |3J). However, the latter representation is better from the 
computational viewpoint (see [6]), and the main point is that both descriptions of ordinal 
patterns provide exactly the same partitions of the interested system. In particular, both 
representations contain the information on whether f ok (x) < f ol (x) for k, I G {0, 1, . . . , d} 
with I < k. 

Partitions and entropies. Let X C K, let / : X be a B(X)-B(X)-measurable map, 
where M(X) denotes the Borel a-algebra on X, and let \x be an f -invariant probability 
measure on B(X), that is 

for each A G B(Jf). For the following we refer to the simple fact that finite a-algebras 
on X are in one-to-one relation to finite partitions of X, since there is exactly one finite 
partition generating a finite a-algebra. 
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Recall the definition of the (Shannon) entropy of a finite partition A = {Ai,A 2 , 
. . . , Ak\ C M(X) of X and of the from A generated cr-algebra £>, respectively, given 
by 

k 

H(B) = H(A) = - y £»( A i)^»(A j ). 

i=i 

For a finite partition A C M(X) of X, let h^f^A) := lim,^ ±H(Y££ f~ oj {A)), 
where \/™Zo f ~ oj (A) denotes the cr-algebra generated by the set of all f~ i(A) with 
j G {0, 1, . . . , n — 1} and A G A. It is well known that this limit exists (see, e.g., [TO] . 
Cor. 4.9.1). The Kolmogorov-Sinai entropy of / is defined by 

h^f) := swp{h^(f,A) | A C B(X) finite partition of X}. 

The Kolmogorov-Sinai entropy is an important theoretical concept in ergodic theory, how- 
ever, its estimation from an orbit of a dynamical system, is complicated since principally 
arbitrarily fine partitions have to be considered. The permutation entropy introduced by 
Bandt and Pompe [3] seems to be an interesting alternative. 

Definition 2. Let X C R, let / : X be a B(X)-B(X)-measurable map, and let fi be 
an /-invariant probability measure on B(Jf). For d G N, let 

V d = {Pi | i G J4 with Pi = {s G X | = i}. 

The permutation entropy of / is defined by 

h;(f)=hmsup±H(V d ). 

d— >oo A 

Note that for each d G N and i G Z^, the set Pi indeed belongs to M(X). This immediately 
follows from the facts that {x G X | /°( d ~ r )(x) < /^^^(x)} = - f°( d - r )y\X n 

[0, oo[), the mapping - /°( d " r ) is B(X)-B(X)-measurable and X n [0, oo[ G B(X) 

(compare formula ([T])). 

2 Generating properties 

The key for proving the main results of the paper is that in the ergodic case ordinal 
patterns separate points, in other words, the partitions defined by ordinal pattern are 
'generating'. Note that finite generating partitions play an important role in ergodic 
theory (see [10J, §4.6). By considering ordinal patterns of increasing order we obtain 
a sequence of finite partitions Pi, 7-2,^3; • • • generating increasing cr-algebras, that is 
a{Vi) C a(P 2 ) C a(P 3 ) C . . .. 

The following lemma shows how ordinal information of a dynamical system with invariant 
measure can be extracted from its orbits. Recall that if X C R and / : X *-^> is M(X)- 
B(X)-measurable, then an /-invariant probability measure /i on M(X) is called ergodic 
if 

H(A) G {0, 1} for each A G M(X) with / _1 (A) = A. 
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It is well-known that this is equivalent to the statement that fj,(f 1 (B)AB) = for 
B G B(X) implies fx(B) G {0, 1}, where A denotes the symmetric difference of sets (see 
PD], Th. 1.5.). 

Lemma 3. Let X C R ; let f : X ^ be a M(X)-M(X) -measurable map, and let [i be an 
f -invariant probability measure on B(X). 

Then there exists a set X G B(X) with n(X) = 1 and f{X) C X, suc/i £/ia£ 

a(x) := lim d,d ^ - 

d^oo d 

is well-defined for all x G X . If fi is ergodic, X can be chosen such that moreover 
ol(x) = /J>({y G X | y < x}) and ct is injective on X . 

Proof. Let zGl, and for x G X let 

#{zG{l,2,...,4|r(x)<z} 
a(z, xj := lim — - 

if the limit exists. Fix a set X z G B(X) with fi(X z ) = 1 consisting of points x G X for 
which a;) exists. Moreover, in case of ergodicity of / require that fi({y G X\y < 
z}) = ct(z,x) for all x G X z . Both is possible by the Birkhoff Ergodic Theorem (see, 
e.g., [10], Th. 1.14. and text below). For X z : = f|„ gN f~° n {X z ) it holds f(X z ) C X z 
and, since \x is /-invariant, [i{X z ) = 1. Moreover, since ck(z, x) is defined if and only if 
ot(z, f(x)) is, cx(z, x) is defined for all x G X z . 

Let now Z be a countable dense subset of X containing all points of X isolated from the 
left or from the right, that is, every z£l with ]z — e, z[ H X = or ]z, z + e[ H X = 
for some e > 0. Let X = {~\ zeZ X z . Obviously, //(X) = 1, and /(X) C X. We show that 

ct(x, x) is defined for x G X. If x G X D Z, this is obvious. For the case where x G X \ Z, 
consider 

£ = { ] — oo, z] | z G Z} 
and let o~(£) denote the a-algebra on R generated by £. Obviously, 

:= lim #{ i£ {i.2,... d}ir W£B} 

d^oo d 

is well-defined for B & £ and can uniquely be extended to a probability measure on the 
measurable space (R, &(£)), which we still denote by P x . Now let (yk)kem be a mono- 
tonically increasing sequence in Z and (z^ken be a monotonically decreasing sequence 
in Z such that lim^oo yk = lim^oo Zk = x. (By definition of Z such sequences always 
exist.) By definition of X, the numbers a(yk,x),oc(zk,x) are defined for all k G N. Ob- 
viously, the sequences (cx(y k , x))k^n and (a^z*., x))k£N are monotonically nonincreasing 
and monotonically nondecreasing, respectively, therefore the limits lim^oo ct(yk, x) and 
linn^oo ct(zk, x) both exist. Further, P x ({x}) = lim^oo #{' g { 1 ' 2 '---.4 1 / W-^} - IS obviously 
well-defined. In particular, P x ({x}) = — if m is the least integer such that f om (x) = x 
and P x ({x}) = if no such integer exists. Hence 

a{y k ,x) + P x ({x}) < hmmf 

d^co a 

, y #{»£{l,2,...,4|f(x)<x} 

< hmsup < a{Zk,x) 

d— >oo U> 
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for every fceN. By the upper continuity of finite measures one obtains 

lim (ex(z k ,x) - (ex(y k ,x) + P x ({x}))) = lim P x Qy k , z k \) - P x {{x}) = 0. 

Consequently, lim^oo ^^ e ^ 1 ' 2 '---'^ I ( x ^ x } exists, and hence ex(x) = ex(x,x) is defined for 
each x & X and coincides with lim^oo /i({y G X \ y < z k }) = [i({y G X \ y < x}) for ji 
ergodic. 

In particular, ex is monotonically nondecreasing in the ergodic case. Therefore, if J = 
ex' 1 (a) consists of more than one point for some a G K, then J is the intersection of 
X with some interval, hence there are at most countable many of such a. In the case 
/i(J) > 0, necessarily, min J exists and ^(J) = /i({min J}). Otherwise, one would find 
some x,y G J with x < y such that /i( ] — oo, x] H J) < n( ] — oo, y] H J), leading to 

ct(x) — fj,() — oo, x) PI X) < — oo, y] H X) — a(y) 

in contradiction to ex being constant on J. Let 

U = [j{J\ J = ex-\a) for some a G M, #J > 1,/jl(J) = 0} 

U (J{J\ {min J} | J = ex' 1 (a) for some a G R, #J > l ;/ u(J) > 0} 

Clearly, /x(?7) = 0. By omitting all points with iterates in U the set X can be modified 
such that ex is injective. □ 

Remarks: 

1. The proof of the lemma is given under relatively general assumptions. If [i is ergodic, 
then ex is just the distribution function of /i, which can easily seen by the 'ergodic part' 
of the Birkhoff Ergodic Theorem (see, e.g., the remark below Th. 1.14. in [TO]) and the 
Clivenko-Cantelli argument (see, e.g., |9j, sec. II. 2.). It remains to show injectivity of ex 
on a set of full measure, which is the last part of our proof. The case that X is compact 
and / is continuous can reduced to the ergodic case argueing with ergodic decomposition 
(see e.g. Pj, §6.2.) 

2. Bandt and Shiha |2j have used the above statement for ergodic /i with continuous dis- 
tribution function, in order to show that all finite-dimensional distributions of a stationary 
stochastic process can be reconstructed from its ordinal structure and the one- dimensional 
distribution. 

As the above lemma shows, ergodic case ordinal patterns are 'generating' in the ergodic 
case: 

Corollary 4. Let IcR, let f : X be a M(X)-M(X) -measurable map, and let be an 
f -invariant ergodic probability measure on M(X). 

Then there exists a set X C X with n(X) = 1 and f(X) C X , such that different points 
of X have different ordinal patterns of some order d. 

We will need the supplement to Lemma [3] below, where \f deN Vd denotes the cr-algebra on 
X generated by \J deN V d - 
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Lemma 5. Let X be defined as in Lemma\^ ex be considered as a map on X, and V 
denote the restriction of \J d£N Vd to X. Then ex is V-M([0, 1}) -measurable. 

Proof. For a G [0, 1] and X a := {x G X \ ex(x) < a} it holds 

X a = {x G X | For all q G Q H ]x , oo[ there exists a d G N 

with - < q for all d > d } 
d 

= I f| U P) {x G X | z d , d (x) < dq} J n X. 

Since for each G N, q G Q , the set {x G X | id,d(x) < dq} belongs to the cr-algebra 
generated by V d , the set r\ 6 Qn]* ,°o[ Ud () eN fld>d { x e ^ I < d <?} is contained 

in \J d€N Vd- The system {]0, a],|a G [0,1]} generates B([0, 1]), and hence the result 
follows. □ 



3 Kolmogorov- Sinai entropy and order structure 

As well known, for a map / and an invariant measure fi the Kolmogorov-Sinai entropy can 
be obtained as lim^oo h^(f, A n ) if M(X) is generated by a sequence of finite partitions 
A n with corresponding increasing a-algebras (see [10], Th. 4.22). The following theorem 
says that under relative mild assumptions this sequence can be chosen in a standard way, 
namely by considering the partitions of X with respect to ordinal patterns of order d for 
increasing d. 

Theorem 6. Let X cl, let f : X be a M(X)-M(X) -measurable map, and let fi be an 
f -invariant ergodic probability measure on M(X). If /i is ergodic or X is compact and f 
continuous, then 

(2) K{f) = Jim h^f,V d ) = sup h^f,V d ). 

Proof. We start with the case that fi is ergodic. Clearly, \/ d€N Vd is contained in M(X) 
(see end of Section [1]). Let X be chosen as for the ergodic case in Lemma [3] and let ex be 
considered as a map on X. Then ex is injective, monotone and, by LemmaO P-B([0, Im- 
measurable. Therefore, one easily sees that V coincides with the restriction of M(X) to 
X. Hence for each B G M(X) there exists an A G \/ den Vd with fj,(AAB) = 0, and © 
follows from Theorem 4.22 in [ID] . 

In the case that /i is non-ergodic use ergodic decomposition (see, e.g., [ID], §6.2.) and the 
Monotone Convergence Theorem (see e.g. [ID], Theorem 0.8): fi can be written as /i = 
J E mdT(m), where E denotes the set of ergodic /-invariant Borel probability measures 
and r is a Borel measure on the /-invariant Borel probability measures on X (with respect 
to the weak* topology) with t(E) = 1. According to results shown by Jacobs (compare 
[ID] . Theorem 8.4), it holds 

(3) h^if, A)= I h m (f, A) dr(m) for each finite partition A C B(X) of X, 

Je 

(4) K{f)= I h m (f)dr(m). 

Je 
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By the already shown, the sequence (gdjdm of functions g d on the /-invariant Borel 
probability measures defined by gdijn) = h m (f,Vd) on E and vanishing outside of E 
is monotonically increasing and converges pointwisely to h m (f) on E. By the Monotone 
Convergence Theorem, either (h^f, V d ))deN is bounded and it holds (J2J), or (h^f, Vd))d<=N 
is unbounded and h m (f) is infinite on a set of positive measure with respect to r. In the 
latter case h^(f) = 00. □ 



4 Permutation entropy 

As a relatively simple consequence of the above theorem, we obtain that the Kolmogorov- 
Sinai entropy is not larger than the permutation entropy. This generalizes a result given 
by Amigo et al. [I] for ergodic maps / on intervals with finite h^(f). 

Corollary 7. Let let, let f : X be a B(X)-B(X) -measurable map, and let /i be an 
f -invariant ergodic probability measure on M(X). If \x is ergodic or X is compact and f 
is continuous, then h^(f) < h^(f). 

Proof. If h^f) = 0, we are done. Otherwise, consider some a > with h^f) > a. 
Given some c > 1 with > ca, by the above theorem there exists ado^N with the 

following property: For each d > do there exist some n{d) with ^i^(Vj=o /~°' 3 'C^ > d)) > ca 
for all n > n(d). 

Let d > do and n > max{ra(d), -^j}- Note that the cr-algebra generated by "P^+n contains 
Vj=o f~°^(f > d)- This can be seen from the fact that for each i = (ii, i 2 , . . . , id) G Id and 
j G {0, 1, . . . , n — 1} one has 

d d 

r oj (Pi) = f]{xeX\ id.i (/ ■'(■'■)) = »,} = P|{x G X I i w (a;) = it}. 
1=1 1=1 

Therefore, one easily sees that f~°^(Pi) is contained in the cr-algebra generated by sets 
{x G X I f ok (x) < f ol {x)} with k, I G {0, 1, . . . , d + n} and I < k (compare to §B)). Since 
validity of f ok (x) < f (x) only depends on the ordinal pattern of x (see below definition 
of ordinal patterns), such sets are the union of sets in Vd+ n , hence the set f~°^(Pi) belongs 
to the ex-algebra generated by Vd+ n - 

Consequently, H(V d+Tl ) > i^(V™=o f~° J {Pd)), which implies 

1 1 n ~ x 

ITU £ IT-n^Vf-"™ 

> 7 — T^—myr^CPd)) 

(c — 1) n + n v 

v 1 ^ 3=0 



cn v 



3=0 
> a. 



From this one obtains h*(f) > a and, since a can be chosen arbitrarily near to h^f), it 
follows h*Jf) > h^f). □ 
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The central question under which assumptions Kolmogorov-Sinai entropy and permuta- 
tion entropy coincide remains open in the general case, whereas for piecewise monotone 
interval maps with invariant probability measure coincidence has been shown by Bandt 
et al. [5], as already mentioned in the Introduction. A map / from an interval [a, b] is said 
to be piecewise monotone if there are points Cq, c\, . . . , with a = c < C\ < . . . < Ck = b 
such that / is continuous and strictly monotone on [cj_i,Cj] for each j = 1,2,..., A;. 
(Continuity of the whole map is not required, in contrast to the often given definition of 
a piecewise monotone interval map.) Note that the proofs in [5] do not provide an idea 
for a generalization since they essentially base on piecewise monotonicity, in particular, 
on a result given by Misiurewicz and Szlenk [8] . 

As mentioned in the Introduction, Amigo et al. [1] have given a modified concept of permu- 
tation entropy of an interval map, for which they have shown equality to the Kolmogorov- 
Sinai entropy. Their idea is to consider permutation entropy for a stationary stochastic 
process with values in a finite set equipped with an arbitrary order. (The resulting en- 
tropy does not depend on the order.) Given a partition of the interval on which the map 
is defined into finitely many subintervals, one only regards in which of the subintervals a 
point lies. This yields the desired stochastic process, and the permutation entropy of the 
particular partition is naturally defined. The permutation entropy of the interval map 
is then defined as the limit of permutation entropies of partitions with interval lengths 
tending to 0. 

Just like the Kolmogorov-Sinai entropy, this modified concept of permutation entropy 
uses infinitely many partitions which in contrast to the original definition of permutation 
entropy are non-standardized. Note that the result of Amigo et al. p] is given not only 
for interval maps but in a multidimensional context. 
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